Journal of Artificial Intelligence Research 12 (2000) 93-103 



Submitted 11/99; published 3/00 



Exact Phase Transitions in 
Random Constraint Satisfaction Problems 



Ke Xu 
Wei Li 

National Laboratory of Software Development Environment, 
Department of Computer Science and Engineering, 
Beijing University of Aeronautics and Astronautics, 
Beijing, 100083, P.R. China 



KEXU@NLSDE. BUAA.EDU.cn 
LIWEI@NLSDE. BUAA.EDU.cn 



Abstract 



In this paper we propose a new type of random CSP model, called Model RB, which is 
a revision to the standard Model B. It is proved that phase transitions from a region where 
almost all problems are satisfiable to a region where almost all problems are unsatisfiable 
do exist for Model RB as the number of variables approaches infinity. Moreover, the critical 
values at which the phase transitions occur arc also known exactly. By relating the hardness 
of Model RB to Model B, it is shown that there exist a lot of hard instances in Model RB. 

1. Introduction 

Since the seminal paper of Cheeseman, Kanefsky and Taylor (1991) appeared, there has 
been a great amount of interest in the study of phase transitions in NP-complete problems. 
However, it seems to be very difficult to prove the existence of this phenomenon or to obtain 
the exact location of the transition points for such problems. For example, in random 3- 
SAT, it is known from experiments that the phase transition will occur when the ratio of 
clauses to variables is approximately 4.3 (Mitchell, Selman, &: Levesque, 1992). Another 
experimental estimate of the transition point suggested by Kirkpatrick and Selman (1994) 
is 4.17. They used finite-size scaling methods from statistical physics to derive the result. In 
contrast with the experimental studies, the theoretical work has only given some loose but 
hard won bounds on the location of the transition point. Currently, the best known lower 
bound and upper bound are 3.003 (Frieze & Suen, 1996) and 4.602 (Kirousis et al., 1998) 
respectively. Recently, Friedgut (1999) made tremendous progress towards establishing the 
existence of a threshold for random A;-S AT by proving that the width of the transition region 
narrows as the number of variables increases. But we still can not obtain the exact location 
of the phase transition point from this approach. 

In fact, SAT is a special case of the constraint satisfaction problem (CSP). CSP has not 
only important theoretical value in artificial intelligence, but also many immediate appli- 
cations in areas ranging from vision, language comprehension to scheduling and diagnosis 
(Dechter, 1998). In general, CSP tasks are computationally intractable (NP-hard) (Dechter, 
1998). In recent years random constraint satisfaction problems have also received great at- 
tention, both from an experimental and a theoretical point of view (Achlioptas et al., 1999; 
Cheeseman et al., 1991; Frost &: Dechter, 1994; Gent et al., 1999; Hogg, 1996; Larrosa & 
Meseguer, 1996; Prosser, 1996; Purdom, 1997; Smith h Dyer, 1996; Smith, 1999; Williams 
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Sz Hogg, 1994). Williams and Hogg (1994) developed a technique to predict where the 
hardest problems are to be found and where the fluctuations in difficulty are greatest in a 
space of problem instances. They have also shown that their predictions of the critical value 
agree well with the experimental data. Smith and Dyer (1996) studied the location of the 
phase transition in binary constraint satisfaction problems and discussed the accuracy of a 
prediction based on the expected number of solutions. Their results show that the variance 
of the number of solutions can be used to set bounds on the phase transition and to indicate 
the accuracy of the prediction. Recently, a theoretical result by Achlioptas et al. (1999) 
shows that many models commonly used for generating random CSP instances do not have 
an asymptotic threshold due to the presence of flawed variables. More recently, Gent et al. 
(1999) have shown how to introduce structure into the conflict matrix to eliminate flaws. 

In this paper we propose a new type of random CSP model, called Model RB, which 
is a revision to the standard Model B (Gent et al., 1999; Smith & Dyer, 1996). It is 
proved that the phase transition phenomenon does exist for Model RB as the number of 

variables approaches inflnity. More precisely, there exist two control parameters r, p and 
the corresponding critical values rcj^, Pcr such that for each fixed value v <i v^^^ or p <^ Pcr^ *^ 
random CSP instance generated following Model RB is satisfiable with probability tending 
to 1 as the number of variables approaches infinity, and when r > Vcr ox p > p^r, unsatisfiable 
with probability tending to 1. Moreover, the critical values and Pcr are also known 
exactly. By relating the hardness of Model RB to Model B, it is shown that Model RB 
actually has a lot of hard instances. 

2. Definitions and Notations 

A constraint satisfaction problem (CSP) consists of a finite set U = {ui, ■ ■ ■ ,Un} of n 
variables and a set of constraints. For each variable Ui, a domain Di with di elements 
is specified; a variable can only be assigned a value from its domain. For 2 < k < n a. 
constraint Cii^i2^...^ik consists of a subset {itji, 1*225 ■ ■ ■ ^uik} of U and a relation Rii^i2,-,ik ^ 
Dii X • • • X Dik, where il, i2, - ■ ■ ,ik are distinct. C.ri^.j2,---,ifc is called a A;-ary constraint which 
bounds the variables un, ■ ■ ■ ,Uii~- Rii,i2,—,ik specifies all the allowed tuples of values for 
the variables un, - ■ ■ ,Uik which are compatible with each other. A solution to a CSP is an 
assignment of a value to each variable from its domain such that all the constraints are 
satisfied. A constraint Cii..j2,---,iA; is satisfied if the tuple of values assigned to the variables 
Uii, - ■ ■ ,Uik is in the relation Rii^i2,---,ik ■ A CSP that has a solution is called satisfiable; 
otherwise it is unsatisfiable. In this paper, the probability of a random CSP instance being 
satisfiable is denoted by Pr{Sat). 

We assume that k > 2 and all the variable domains contain the same number of values 
d = n" in Model RB (where a is a constant). The generation of a random CSP instance in 
Model RB is done in the following two steps: 

Step 1. We select with repetition t = rnlnn random constraints. Each random constraint 
is formed by selecting without repetition k of n variables. 

Step 2. For each constraint we uniformly select without repetition q = p ■ d^ incompatible 
tuples of values, i.e., each constraint relation contains exactly (1 —p)-d^ compatible tuples 
of values. 
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The parameter r determines how many constraints are in a CSP instance, while p 
determines how restrictive the constraints are. 

The following definitions will be needed in section 4 when we derive the expectation of 
the second moment of the number of solutions. 

Definition 1 An assignment pair is an ordered pair {ti,tj) of assignments to the variables 
in U, where ti = {an, ai2, • • • , dm) dnd tj = (flji, aj2, • • • , ajn) with au^aji G Di. An assign- 
ment pair {ti,tj) satisfies a CSP if and only if both ti and tj satisfy this CSP. The set that 
consists of all the assignment pairs is denoted by Apair- 

Definition 2 Similarity number S-l^ : Apair ^ {0, 1, 2, ■ ■ ■}, 

n 

S^{{ti,tj)) = ^Sam{aii,aji) (1) 
where the function Sam is defined as follows: 

Sam{au,a,0 = [l ^Z^Z 

The similarity number of an assignment pair is equal to the number of variables at which 
the two assignments of this assignment pair take the identical values. By Definition 2 it is 
easy to see that < S^{{ti,tj)) < n. 

3. Main Results 

In this paper, the following theorems are proved. 

Theorem 1 Let r^r = ~ in("-p) • U <^ > < p < 1 are two constants and k, p satisfy the 
inequality k > j^, then 

lim Pr{Sat) = 1 when r < Vcr (3) 

n— >oo 

lim Pr{Sat) = when r > Vcr (4) 

n— >oo 

Theorem 2 Let pcr = 1 — e^~ . If a > ^, r > are two constants and k, a and r satisfy 
the inequality ke^~ > 1, then 

lim Pr{Sat) = 1 when p < pcr (5) 

n— >oo 

lim Pr{Sat) = when p > Pcr (6) 

n— >oo 

4. Proof of Theorem 1 and Theorem 2 

The expected number of solutions E{N) for model RB is given by 

E{N) = -p)™'"" = n""(l -p)™'"" (7) 

By the Markov inequality Pr{Sat) < E{N) it is not hard to show that lim„_).oo Pr{Sat) = 
when r > rcr- or p > pcr- Hence relations (4), (6) are proved. It is also easy to see that 
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E{N) is eqal to 1 when r = or p = pcr, and E{N) grows exponentially with n when 

r < or p < Per- 

The key point in the proof of relations (3), (5) is to derive the expectation of the second 
moment E{N'^) and give an asymptotic estimate of it. Let be a random CSP instance 
generated following Model RB. P{{ti,tj)) stands for the probability of {ti,tj) satisfying 0. 
Now we start to derive the expression of P{{ti,tj)). Since each constraint is generated 
independently, we only need to consider the probability of {ti,tj) satisfying a random con- 
straint. Assuming that the similarity number of {ti,tj) is equal to 5, we have the following 
two cases: 

(1) Each variable of a constraint is assigned the same value in tj as that in tj. In this 

(d^ — 1 \ / \ 
j/y j. 

(d^ — 2 \ / d^ \ 
j/y j. 

The probability that a random constraint falls into the first case is^^j/^^j. Hence 
the probability into the second case isl— ^^^/^^^ Thus we get 



Pi{U,tj)) 



a'r) it) i 



s 



S -2 



\ 



(t) il) (T) ■ Hi} 



it) 



rn In n 



(8) 



Let Ag be the set of assignment pairs whose similarity number is equal to 5 . It is easy 
to show that the cardinality of Ag is given by 



\As\ = d-[ I )(d-ir 

Prom the definition of E{N'^), we have 

n 



(9) 



s=o 



<r(")«i 



\n-S 



a*-') it) ( 



df" 



+ 



V 



) C) C) 



), it) 



\ 



(10) 



It is very difficult to analyze the above expression directly. First, we give an asymptotic 
estimate of P{{ti,tj)). Let s = ^. It is obvious that < s < 1. By asymptotic analysis, we 
get 

( S^ 



n '■ 



n n' n> 



il) 



(l_i)(l_ 2)...(i_ fc^) + n ^^^„2^ 
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where 



and 



9is) 



k{k - l)(s* - s*-i) 



(V) 
(0 



1-p 



f (fi - 2 \ 

V q ) (d'' - q){d'' - q - 1) ,^ „,1 



( : ) 

Note that d = n", we have 



d^{d^ - 1) 



(l-p)-(s* + ^) + (l-p)2-(l-s* 



By use of the condition a > ^, we get 



(11) 



(12) 



(13) 



7'ri In n 



i + -^(.* + ^) 

1 — P n 



rn In n 



(i + 0(r)) 



(14) 



(15) 



For every < s < 1 (where s = f ), the asymptotic estimate of \As\ i 



IS 



1^^ 



i/27rns(l — s) 



gn(-slns-(l-s)ln(l-s))^-|^ _(_ (9(1)) 

I n 



'n" -y27rns(l - s) 
Notice that E{N) = n""(l -|,)™i°", we have 

(i->-(>^(:j (!+<)) (17) 



|As|P((ii,ij))=S^(A^) 



l + -^(.^ + ^) 
I — p n 



rn In n 



When n is sufficiently large, except the first term E^{N), \As\P{{ti,tj)) is mainly de- 
termined by the following terms: 



T rn In n 



( — )' 



We can rewrite it as 
Let 



1-p 



/i(s) = rln(l + ^-s'=) 

1 -p 



as 



The second derivative of h{s) is 

h"{s) : 



rA;ps*^ ^[(A; - 1)(1 -p) -ps* 
(1 — p + ps^Y 



(18) 

(19) 
(20) 

(21) 
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Applying the condition k > in Theorem 1 to the above equation we can easily prove 

that h"(s) > on the interval < s < 1. For Theorem 2, from the condition ke~~ > 1 it 
follows that the inequality k > still holds when p < pcr- It is also not hard to show that 
h{0) = 0, and h{l) = — r ln(l — p) — a < when r < or p < pcr- Hence we can easily 
prove that the unique maximum point of h{s) is s = when r < or p < pcr ■ Thus the 
terms of < s < 1 are negligible when r < r^r or p < prr ■ We only need to consider those 
terms near s = . The process can be divided into the following three cases: 

Case 1: a > 1. When 5 = (s = 0), from the definition of g{s) in Equation (11) we 
have 



1 — p n _ 



(22) 



Thus by Equation (17) we get 



As\P{{U,tj)) ^ E\N){1 - ^ E\N) (23) 



n 

When 5=1 {s = it also not hard to prove that 



rn Inn 

•It , / / I W I 

lim 



1 + — ^ s + 
1 — p n 



e° = 1 (24) 



Hence we obtain 



Similary, 



As\P{{ti,tj)) ^ E\N)n'-'^ when S = 1 (25) 



\As\P{{ti,t^)) « E\N)^^ when S 



^3(l-a) 

"3! 

Summing the above terms together, we obtain 



As\P{{ti,tj)) ^ E''{N)^;^ when 5 = 3,--- (26) 



EiN-") = \As\P{{ti,tj)) ^ E^{N)e-'-'' ^ E^{N) (27) 

Case 2: a = 1. By use of the method in Case 1, it can be easily shown that 

\As\Pi{ti,tA) ^ E'^{N){1 - -)" ^ E^{N)- when 5 = 

n e 

\As\Pi{ti,tj)) ^ E^{N)^ when 5 = 1 
\As\Pi{ti,tj)) ^ E^iN)-^^ when 5 = 2 
\As\P{{ti,tj)) ^ E\N)-^^ when5 = 3,--- (28) 
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Summing the above terms together, we obtain 

n . 

E{N^) = J2 \As\P{{ti,tj)) ^ E'^iN)- . e = E\N) (29) 

5=0 ^ 

Case 3: ^ < a < 1. Let Sq = (where ^ is a constant and satisfies 1 — Q!<^<1 — |). 

It is not hard to show that when 0<5<5o(0<s< n^^^ < n~'k), the following limit 
holds: 

lim ^-(/ + ^) ■ nlnn = (30) 

n->-oo I — p n 

Thus when < 5 < 5o, the asymptotic estimate of the second term in the right of Equation 
(17) is 



1 + -^—{s" + ' 



rnlnn 



e'' = 1 when n ^ oo (31) 



I — p n 

So when Q < S < Sq, the asymptotic estimate of \As\P{{ti,tj)) is 

\As\P{{U,t,)) ^ E\N){ ^ )(1 - ^r-'i:^)' (32) 

It should be noted that ^ ^ j(l — ■^)"'~^ {^)^ is a binomial term whose maximum point 
is around S = n^~", and Sq = > n^~". By asymptotic analysis, we obtain 

V f ^ )(i - J-)-^(J-)^ « E f c )(i - —r-'{—f = 1 (33) 

Thus we get 

n 

E{N') = J2 \As\P{{U,tj)) ^ E\N) (34) 

Combining the above three cases gives 

E{N'^) Ri E'^{N) when r < or when p < pcr (35) 

Hence 

E'^{N) 

lirn^ ^^j^^2'j ~ ^ ^^^^ f < '"cr or when p < pcr (36) 



71— ^OO 



By the Cauchy inequality Pr{Sat) > g^^^ (BoUobas, 1985), it can be easily proved 
that linin^oo Pf (Sat) = 1 when r < or p < pcr- Hence Theorem 1 and Theorem 2 are 
proved. 
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5. The Relation between Model B and Model RB 

In this section we will explain in detail how Model RB revises Model B and show the 
hardness of Model RB by relating it to Model B. Prom the previous papers (Gent et al., 
1999: Smith &: Dyer, 1996) we know that the generation of a random CSP instance in the 
standard Model B (which is often written as (ra, d,pi,p2)) is done in the following two steps: 
Step 1. We select with repetition t = pi random constraints. Each random constraint 

is formed by selecting without repetition 2 of n variables. 

Step 2. For each constraint we uniformly select without repetition q = p2 ■ d"^ incompatible 
tuples of values, i.e., each constraint relation contains exactly (1 — P2) ' d^ compatible tuples 
of values. 

Since the standard Model B is a binary CSP model, we will only consider the binary 
case of Model RB in this section. In the previous papers Model B was used to test the 
CSP algorithms in the following way. Given the values of n, d and pi, the constraint 
tightness p2 was varied from to 1 in steps of ^. At each setting of {n,d,pi,p2) a fixed 
number of instances (e.g. 100) were generated. The search algorithm was then applied 
to each instance. Finally numerous statistics about the search cost and the probability of 
being satisfiable were gathered. In fact, the two steps of forming a constraint and selecting 
incompatible tuples of values in Model RB is exactly the same as those in Model B. The 
significant difference between Model B and Model RB is that Model RB restricts how fast 
the domain size and the number of constraints increase with the number of variables while 
Model B does not, which may lead to the result that many instances of Model B suffer from 
being asymptotically trivially insoluble (Achlioptas et al., 1999) while Model RB avoids this 
problem. But it is easy to see that given the values of n, d andpi, for the setting (n, c?,pi,p2) 
of Model B there is an equivalent setting in Model RB with the same number of variables 
as that in {n,d,pi,p2), a = |^ and r = ^^2^~^^ (Let n" = d and rnlnn = ^pin{n — 1)). 

Theorem 2 shows that if a > | and 2e~~ > 1, then there exists an exact phase transition 
in the binary case of Model RB. Given the values of n, d and pi in Model B, for the setting 
of {n,d,pi,p2), the conditions that the equivalent setting in Model RB satisfies Theorem 2 
are 

a = > - ^ d^ > n (37) 

Inn 2 ^ ' 

a _ind. 2 In 71 2 lu d , , 

2e"~ > 1 ^ 2e '"'^ > 1 ^ pi > — (38) 

- (n - l)ln2 ^ ' 

The proof of Theorem 2 reveals that if the conditions (37), (38) are satisfied, then Model 
RB will exhibit an exact phase transition at E{N) = 1. It should be noted that Williams 
and Hogg (Williams & Hogg, 1994), and independently Smith (1996) have already developed 
a theory to predict the phase transition point in Model B on the basis of E{N) = 1. Prosser 
(1996) found that this theory is in close agreement with the empirical results, except when 
pi is small. Inequality (38) shows that in order to make the equivalent setting in Model RB 
satisfy the conditions of Theorem 2, the parameter pi in Model B should not be less than 
a certain value, which is consistent with Prosser's experimental finding. 

Now we consider a typical setting (20, 10,0. 6,^2) of Model B. Let n = 20, a = ^ 
0.7686 and r = ^4Sr^ ~ 1-5856 in Model RB. Then the setting of Model RB with such 
values is equivalent to the setting (20, 10, 0. 6,^2) of Model B. Prom Inequalities (37) and 
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(38) it is also not hard to show that the equivalent setting in Model RB corresponding to 
the setting (20, 10, 0.5, j;2) satisfies the conditions of Theorem 2, i.e., 10^ > 20 and pi = 
^■^ — (20-'i{in2 ~ 0.35. The experiments done by Prosser (1996) show that the instances 
generated at no = 0.38 are very hard to solve. This maximum cost point also agrees well with 

_a 0.7686 

the asymptotic phase transition point of Model RB that isp=l — e r l — e 1 5856 pa 0.38. 
For some other settings of Model B in the previous work, we can also find their equivalent 
settings in Model RB using this method. Thus the hardness of solving these settings of 
Model B is equal to that of solving their equivalent settings in Model RB. Prom many 
previous studies (Gent et al., 1999; Smith & Dyer, 1996; Prosser, 1996) we know that the 
instances generated at the phase transition in many settings of Model B are very hard to 
solve for various kinds of CSP algorithms. So there exist a lot of hard instances to solve in 
Model RB. 

6. Conclusions and Future Work 

A lot of experimental and theoretical studies indicate that a phase transition in solvability 
is a very important feature of many decision problems in computer science. It is shown 
that these problems can be characterized by a control parameter in such a way that the 
space of problem instances is divided into two regions: the under-constrained region where 
almost all problems have many solutions, and the over-constrained region where almost all 
problems have no solutions, with a sharp transition between them. Another interesting 
feature associated with the phase transition is that the peak in liardness of solving the 
problem instances occurs in the transition region. Since these instances generated in the 
transition region appear hardest to solve, they are commonly used as a benchmark for 
algorithms for many NP-complete problems. But unfortunately, except for the Hamiltonian 
cycle problem (wliicli is NP-complete), all tlie decision problems that liave exact results 
about the existence and the location of the phase transition are in P class (Parkes, 1997), 
e.g. random 2-SAT. These problems are not so interesting as the NP-complete problems 
from a complexity theoretic point of view because they can be solved in polynomial time. For 
the Hamiltonian cycle problem, using an improved backtrack algorithm with sophisticated 
pruning techniques, Vandegriend and Culberson (1998) recently found that the problem 
instances in the phase transition region are not hard to solve. 

In this paper we proposed a new type of random CSP model. Model RB, which is a 
revision to the standard Model B, and the asymptotic analysis of this model has also been 
presented. The results are quite surprising. We can not only prove the existence of phase 
transitions in this model but also know the location of transition points exactly. It was 
further shown that there exist a lot of hard instances in Model RB by relating its hardness 
to the standard Model B. Since there is still some lack of studies about the exact derivation 
of the phase transitions in NP-complete problems, this paper may provide some new insight 
into this field. However, we did not discuss the scaling behaviour of Model RB and some 
other related issues in this paper. In order to get a better understanding of Model RB, 
we suggest that future work should include determining either empirically or theoretically 
whether or not hard instances persist with reasonably high frequency as the number of 
variables increases. ^ 

1. Two anonymous referees suggest this point. 
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